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ABSTRACT 

The Commission on Higher Education conducted a survey 
in 1995 of 495 member institutions of higher education, most in the 
Mid-Atlantic region, to determine their progress with outcomes 
assessment. Sixty-eight percent of the institutions responded, 
disclosing that: (1) more than half do not have an institution-wide 

plan for outcomes assessment though more than one-third have had such 
a plan for 3 or more years; (2) most institutions with a plan 
incorporate direct measures of institutional effectiveness, but few 
emphasize direct measures of teaching and learning; (3) the majority 
of institutions without a plan have completed no more than two or 
three of the nine steps considered prerequisite to developing a plan; 
(4) faculty and academic administrators tend to be involved in, 
respectively, developing assessment plans and continuous 
administration of those plans; (5) faculty resistance and a lack of 
sufficient human and financial resources are the chief barriers to 
implementing new assessment activities or sustaining existing 
initiatives; and (6) the Commission could be most helpful by 
publishing guidelines and conducting workshops and seminars to 
provide examples of effective assessment programs and strategies. 
(Author/MAH) 
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Abstract 

The Commission on Higher Education, Middle States Association of Colleges and Schools, conducted a 
survey in 1995 of 495 member institutions of higher education to determine the progress being made 
with outcomes assessment in the region, and 68 percent of the institutions responded. Over one-half of 
the respondents (57 percent) do not have an institution-wide plan for outcomes assessment, although 
over one-third (34 percent) have had one for three or more years. Most institutions with a plan 
incorporate direct measures of institutional effectiveness, but few emphasize direct measures of teaching 
and learning. The majority of institutions without a plan have completed no more than two or three of 
the nine steps considered prerequisite to developing a plan. Faculty tend to be fully involved in 
developing assessment plans at most institutions, while academic administrators tend to be fully involved 
in the continuous administration of those plans. However, faculty resistance and a lack of sufficient 
human and financial resources are the chief barriers to implementing new assessment activities or 
sustaining existing initiatives. Respondents also indicated that the Commission could be most helpful by 
publishing guidelines and by conducting workshops and seminars to provide examples of effective 
assessment programs and strategies. 
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Outcomes Assessment in the Middle States Region 

A Report on the 1995 Outcomes Assessment Survey 

(July 1996) 



A survey was mailed in May 1995 to the chief executive officers of the 495 colleges and 
universities that are members of the Middle States Association of Colleges and Schools. 

Of these institutions, 337 (68 percent) responded to the survey. 

This preliminary report describes the respondents, the extent to which asssessment plans exist in 
the region, the measures and instruments that institutions utilize, institutional readiness to 
develop a plan, the ways in which institutions utilize asssessment findings, and the participants in 
the assessment process. Two types of questions solicited open-ended responses which are also 
analyzed in this report, one identifying difficulties or barriers faced in institutionalizing a plan 
and the other asking how the Commission might best assist institutions. 

Respondents and Their Plans 

The survey identified institutions by Carnegie type, size of enrollment, and public or private 
sponsorship, and it identified respondents by position title. The survey also was designed in two 
parts: one requested data on the information needs of institutions that have not yet developed an 
outcomes assessment plan, and the other examined several aspects of assessment at institutions 
that have implemented a formal program. 

Respondents were broadly representative of the seven Carnegie types adopted by the 
Commission for its purposes in the Middle States region. Respondents in each type 
approximated 68 percent of the total number of institutions in the corresponding type for the 
region as a whole. They were similarly representative of the public (67 percent) and independent 
(66 percent) sectors. 

Over one-half of the institutions (57 percent) have no outcomes assessment plan (Figure 1). 
However, 1 8 percent of the institutions have had a plan for five years or more, 1 6 percent have 
operated with a plan for three to four years, and 9 percent have had a plan for one to two years. 

Public institutions were more likely to have an outcomes assessment plan, representing 54 
percent of the sample. In contrast, only 34 percent of the independent institutions in the sample 
had a plan. 
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The preponderance of public institutions among those that had a plan does not necessarily mean 
that assessment plans exist because of a state requirement. In fact, the responses to the question 
asking if their state required assessment were inconclusive. Approximately 45 percent of the 
public institutions that have a plan and 69 percent of the private institutions indicated that their 
state did not require assessment. In addition, most of the states identified as requiring assessment 
appear on the list of states identified by others as not requiring assessment. It also is not clear 
that the requirement applies only to certain types of institutions, since most of the seven Carnegie 
types used in the survey appear on both lists. It is possible, therefore, that the individuals who 
responded to the survey simply did not know the correct answer to the question. 



Figure 1 

Institution-wide Assessment Plans 

Percentage of All Institutions 
(n=337) 




No Plan 57% 



The individuals assigned to respond to the survey were fairly evenly divided among those at the 
vice presidential level (25 percent), directors of institutional research and/or planning (29 
percent), and deans (23 percent). Only a few survey respondents were chief executive officers (7 
percent), provosts (9 percent) or department heads and faculty (6 percent). 
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Measures and Instruments 

Most institutions that have an outcomes 
assessment plain utilize some of the more 
direct measures of institutional 
effectiveness, but few emphasize direct 
measures of teaching and learning. 

Direct versus Indirect Measures 

Data are most routinely gathered on 
graduation, retention, and attrition rates 
as well as enrollment, all of which 
typically involve counting the number of 
individuals who are processed through a 
particular institution or groups of 
institutions forming a system of higher 
education (Figure 2). A similarly large 
number of institutions focus on various 
types of client-satisfaction surveys, 
including alumni and students who 
comment on their classroom and various 
other campus experiences. 

To the extent that direct measures of 
teaching and learning are utilized, they 
typically involve licensure examinations, 
nationally normed diagnostic tests, and 
program completion tests. 

Other more direct measures of teaching 
and learning are utilized less often in the 
development of an assessment plan, such 
as examinations in discipline content areas, basic skills, higher order skills, vocational or career 
skills, and student portfolios. 



Figure 2 




Outcomes Data Collected 




Percentage of Institutions with Plans 




(n=143) 




Head Counting 




Graduation/Retention/Attrition 


94 


Enrollment Data 


90 


Prof/Grad School Acceptance Rates 


46 


Data from Articulation Agreements 


23 


Survey Data 




Alumni Surveys 


89 


Student Satisfaction Surveys 


85 


Classroom Surveys 


72 


Campus Climate Surveys 


66 


Values/Ethics Surveys 


22 


Student Motivation Surveys 


18 


Test Data 




Licensure Examinations 


73 


Nationally Normed Diagnostic Tests 


64 


Program Completion Tests 


60 


Locally Designed Diagnostic Tests 


58 


Skills Tests 


48 


Locally Designed Remediation Follow-up 


27 


Locally Designed General Skills 


30 


Nationally Normed Remediation Follow-up 


27 


Nationally Normed General Skills 


22 


Other 




Exit Interviews 


62 


Portfolio Assessment 


47 


Case Studies 


12 
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National versus Local Tests 

The survey results also indicated that institutions without an outcomes assessment plan 
nevertheless do utilize some assessment instruments. It specifically asked institutions to indicate 
whether they utilized nationally-normed and locally-developed tests and, if not, whether they are 
considering using or eliminating them, or whether they simply do not use them. The data 
indicate that while institutions frequently rely on nationally-normed tests, a number of them have 
significant experience with and interest in locally-developed instruments (Figure 3). 



Figure 3 

Use of National and Local Tests 

Percentage of Institutions without Plan 
(n=192) 



Nationally Normed Locally Developed 




1995 Outcomes Assessment Survey, CHE/MSA 



Before a plan is in place, a greater number of institutions currently use nationally-normed tests 
(38 percent) than locally-developed tests (29 percent). However, more institutions are in the 
process of actually developing tests locally (52 percent) than are at the stage of considering 
whether or not to use nationally-normed tests (23 percent), which indicates that institutions 
without a plan have a significant interest in locally-developed instruments. 

The interest in certain types of locally-developed tests also is strong once a plan is in place. 

At institutions with a plan, the interest in locally-developed instruments is emphasized for 
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measuring remediation follow-up (27 percent nationally-normed v. 45 percent locally-developed) 
and general skills (22 percent nationally-normed v. 30 percent locally-developed). However, for 
diagnostic purposes, institutions tend to rely more heavily on nationally-normed tests than those 
that are locally-developed. 

On the other hand, nearly one-third of the institutions without a plan (32 percent) do not 
currently utilize nationally-normed tests, and 1 8 percent have not started to develop tests locally. 
Only a handful indicated that they will not utilize nationally-normed instruments (3 percent) or 
locally-developed tests or surveys (1 percent). 

While the Commission on Higher Education recognizes that there are significant issues of 
validity and reliability with some direct measures of teaching and learning, the Commission has 
consistently required institutions to develop a plan for outcomes assessment that strives for "the 
effective assessment of institutional, program, and student learning outcomes" ( Characteristics 
of Excellence, 1994, p. 4), utilizing "both quantitative and qualitative data which demonstrate 
congruence between the institution's mission, goals, and objectives and the actual outcomes 
of.. .educational programs and activities" (p. 17). The purpose of such assessment always is to 
improve teaching and learning and to measure overall institutional effectiveness (pp. 16-17). 

Program Review 

The Commission also expects institutions to engage in program review and make the information 
gleaned from this process an important part of any outcomes assessment plan. Therefore, the 
survey asked all institutions, both with and without a plan, about the relationship between regular 
program review and assessment planning. 

In spite of ambiguities in the question on program review that was addressed to institutions 
without a plan, it appears that 39 percent will include outcomes assessment in their plan, and 44 
percent are still considering its inclusion. The questions on program review addressed to 
institutions with a plan were more direct, indicating that 83 percent have included program 
review in their outcomes assessment plan. At these institutions, an average of 65 to 79 percent of 
the programs at each institution have been assessed at least once. 

Research institutions reported that, on average, 80 to 99 percent of their programs had been 
reviewed at least once. Two-year, specialized, comprehensive, and proprietary institutions — in 
that order and with minor differences — had an average of 65 to 79 percent of their programs had 
been reviewed at least once. Liberal arts institutions had reviewed an average of 50 to 64 percent 
of their programs, while doctoral institutions had reviewed the least number: 20 to 34 percent. 
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Readiness to Develop a Plan 

The survey was particularly concerned with the level of effort that institutions without a plan 
have made in nine aspects of assessment which should be completed in order to set the stage for 
developing an assessment plan. These elements include completing an inventory of the 
institution’s operative statements of mission, goals, and objectives; having approved statements 
covering the hierarchy of goals; developing an inventory of the assessment instruments and 
activities already in place in the institution; evaluating how well these instruments and/or 
activities are working; evaluating the extent to which data produced from existing activities or 
instruments have been analyzed and utlized; deciding what areas should be measured beyond 
those already evaluated; deciding who will respond to the tests or surveys (e.g. students/alumni; 
all/random sample, etc.); deciding whether to include program review in the assessment plan; 
and deciding how and when the test data or survey results are to be processed and analyzed. 

The principal finding is that while institutions have completed some of the more mechanical 
tasks, such as developing inventories of instruments in use, they tend not to have evaluated the 
effectiveness of using those instruments (Figure 4). In addition, while they may have decided 
that program reviews will be included in any future assessment plan, they have not tackled the 
more thorny question of what outcomes still need to be measured, beyond those that are already 
being measured. 
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Figure 4 

Essential Steps in Preparation for 
Developing an Assessment Plan 

Percentage of Institutions without A Plan 
(n=192) 

Completed In Have not 

Progress Started 





n 


% 


# 


% 


# 


% 


Have an inventory of operative statements 


89 


46.4 


90 


46.9 


13 


6.8 


Have decided whether to include program review 


75 


39.1 


85 


44.3 


32 


16.7 


Have an inventory of current assessment instruments 
activities 


65 


33.9 


89 


46.4 


36 


18.8 


Have decided who will respond to tests/surveys 


44 


22.9 


106 


55.2 


41 


21.4 


Have approved statements of goals 
(institutional, program, course) 


39 


20.3 


127 


66.1 


26 


13.5 


Have evaluated the extent to which data from existing 
activities have been analyzed or utilized 


23 


12.0 


96 


50.0 


70 


36.5 


Have decided how/when test results are to be processed 
and analyzed 


20 


10.4 


98 


51.0 


72 


37.5 


Have evaluated how well these instruments/activities work 


18 


9.4 


96 


50.0 


76 


39.6 


Have decided what areas should be measured beyond 
those we already evaluate 


17 


8.9 


109 


56.8 


63 


32.8 
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The six proprietary institutions that do not have a plan, representing 67 percent of all proprietary 
institutions in the region, appear to be in the most advantageous position to develop an 
assessment plan in that they have completed an average of 6.8 of the nine preliminary steps 
(Figure 5). The six research institutions without a plan, which represent 38 percent of all 
research institutions in the region, have completed the least number of steps (1.8) that are 
necessary to begin developing a plan. 



Figure 5 

Number of Preliminary Steps Completed 

[At Institutions without a Plan] 



Type of Institution 

Proprietary 

Specialized 

Two-year 

Doctoral 

Liberal Arts 

Comprehensive 

Research 



Steps Completed 
[Maximum = 9] 

6.8 

3.6 
2.9 

2.7 
2.5* 

2.5* 

1.8 



*Note: While both liberal arts and comprehensive institutions had 
completed the same mean number of steps, liberal arts institutions were 
placed higher on the list because they had a lower standard deviation 
and, therefore, the mean was a slightly more accurate description of the 
group’s status. 



Uses for Assessment Data 

Three questions asked about the ways in which institutions utilized their assessment data. One 
involved the extent to which assessment findings have led the institution to modify its goals and 
objectives, the second was intended to determine if institutions planned to incorporate 
assessment data in the self-study process, and the third inquired about the dissemination of 
findings to the campus community. 
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Modifying Goals and Objectives 

Only a handful (4 percent) indicated that their assessments have led them to modify their goals 
and objectives comprehensively (Figure 6). Most (88 percent) spanned the moderate categories, 
ranging from moderately-to-comprehensively, moderately, and moderately-to-not at all. Only a 
few (9 percent) reported in the absolutely "Not at all" category. 




It could be said that selecting any category of “moderately,” as opposed to “comprehensively,” 
suggests that institutions have certain reservations about the use of assessment data. It is not 
known, however, whether this is simply from a lack of information about instruments and 
strategies that are available or from a lack of confidence in the validity or reliability of existing 
measures as they apply to an institution’s type or its unique mission, goals, and objectives. 
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The Self-Study Process 

Unfortunately, it is not possible to determine if institutions planned to incorporate assessment 
data in their self-study process, because there were ambiguities between the texts of the questions 
and the scope of the response scales, and these responses were discarded. 

Dissemination to the Campus Community 

The question addressed to institutions without a plan also produced ambiguous responses, and 
these data were discarded. However, a different question addressed to institutions with a plan 
was more precise. It asked, in a fixed-response format, how summaries of assessment results are 
communicated to the campus community. A large majority of institutions with a plan (83 
percent) noted that they communicated with the campus by formal report or reports, 59 percent 
did so informally or selectively, and 58 percent did so during their Middle States self-study. 

It is noteworthy, however, that nearly one-quarter (23 percent) of the institutions with a 
plan — representing 10 percent of all institutions in the region — communicated summaries of 
assessment findings only in response to individual requests. The institutions in this group 
represented a diverse range of institutions in the region. They came from six of the seven types 
of institutions and were located in four of the seven states or other political jurisdictions in the 
region, and nearly two-thirds (62 percent) were public institutions. They also were smaller 
institutions: nearly two-thirds (65 percent) had fewer than 5,000 students, and an additional 
21 percent had from 5,001 to 10,000 students. 

This restricted dissemination of assessment results is a clear breach of the letter and spirit of the 
Commission’s standards for accreditation in Characteristics of Excellence, which treat “honesty, 
openness, and concern for.. .constituents” as a reflection on the integrity with which an institution 
“communicates and interacts with its constituencies and the public” ( Characteristics , 1994, 
pp. 5-6). This ethical issue is further reinforced in one of the Commission’s policy statements, 
“Collegiality and Public Communication in the Accreditation Process” {Portfolio of Policy and 
Procedures). While 10 percent of a population may not be a large number for some purposes, its 
weight is far more significant when ethics and integrity are involved. 



Participants in the Process 

Three questions asked respondents at institutions without a plan to estimate the involvement or 
anticipated involvement of faculty, academic administrators, and institutional researchers in 
developing an institution- wide assessment plan. Three additional questions asked institutions 
with a plan to estimate the involvement of these personnel in the continuous administration of 
the plan. A final set of questions for institutions with a plan offered a list of campus constituents 
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and asked who is involved in analyzing assessment results. 

Development vs. Continuous Administration. The data indicate that faculty are fully involved 
in developing assessment plans at a greater number of institutions than they are in the continuous 
administration of plans, where they are more moderately involved (Figure 7). A similar scenario 
applies to academic administrators (Figure 8). However, the role of institutional researchers 
remains fairly stable, whether the institution has or does not have an assessment plan (Figure 9). 

These findings remain consistent when we control for the position title of respondent, indicating 
the presence of a collegial consensus, rather than the particular perspectives of either academic 
administrators, institutional researchers, or those more closely involved with faculty. 

Analysis of Results. The leading group involved in analyzing the results of tests or surveys is 
academic administrators (92 percent of the institutions). They are followed closely by faculty 
committees (83 percent), individual faculty members (74 percent), and institutional researchers 
(72 percent). Least involved in analyzing outcomes are members of the Board of Trustees 
(28 percent), students (12 percent), area employers (8 percent), and alumni/ae (3 percent). 

At the various types of institutions, the distribution of responsibility for analyzing assessment 
results is reasonably consistent region- wide, but there are a few major differences. For example, 
institutional researchers appear to have an extremely low profile at proprietary institutions, 
analyzing the data in only 33 percent of those institutions. The responsibility for analysis at 
proprietary institutions falls on academic administrators and faculty committees, who are 
involved at 100 percent of those institutions. Liberal arts institutions, on the other hand, involve 
institutional researchers slightly less than the average (57 percent) and two-year institutions 
involve them slightly more (83 percent). 

There are also different responsibilities at institutions of various sizes. For example, in small 
institutions, having enrollments of up to 5,000 students, faculty committees and academic 
administrators share most of the responsibility, and institutional researchers have a much smaller 
role (Figure 10). At institutions with more than 15,000 students, the responsibility shifts more 
clearly to academic administrators and institutional researchers, with individual faculty and 
faculty committees having smaller roles. 
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Figure 7 

Perception of Faculty Involvement 
in Outcomes Assessment 
Percentage of Institutions 




1005 Outcomes Aaasaamanl Survey, CHE/MSA 



Figure 9 

Perception of Inst. Research Involvement 
in Outcomes Assessment 
Percentage of Institutions 
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Figure 8 

Perception of Administrators Involvement 
in Outcomes Assessment 
Percentage of Institutions 




1005 Outcomes Assessment Survey, CHE/MSA 



Figure 10 

Involvement in Analyzing Results, 
by Size of Enrollment 
Percentage of Institutions with Plan 

Faculty Acad. Institutional 
Students Faculty Committees Admin. Researchers 
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Barriers to Institutionalizing a Plan 

Faculty and resources, both financial and human, are most frequently cited as the principal 
barriers to implementing an institution-wide plan for outcomes assessment. Other barriers 
include time constraints and a lack of awareness about the value of assessment and effective 
assessment instruments and strategies. 

Two questions solicited open-ended responses identifying the barriers that institutions face and 
how they overcome these barriers. There were 162 responses from institutions without a plan 
and 152 from those that had a plan. 

Institutions without a Plan 

Faculty and resources were the primary barriers noted by institutions without a plan, although 
conflicting priorities and other demands on faculty also were noted. 

Faculty. At least 50 (31 percent) of the responses from these institutions cited faculty as the 
number one barrier to institutionalizing outcomes assessment on campus. This response came 
from public and private institutions of all types and in all states of the region. Illustrative of this 
position was that of a public comprehensive college in Pennsylvania: [There is] "uncertainty and 
resistance — fear that assessment will unfairly measure progress or concentrate too much on 
quantitative data." Faculty anxiety over the potential misuse of assessment data was the concern 
of a public comprehensive college in New Jersey. Yet another view was often heard: 
"Convincing faculty that assessment is not just the most recent educational fad; that it can be 
integrated into the program and promote increased learning. 

Institutions appeared quite candid in their responses. For example, one institution stated that 
most faculty perceive the concept of an institutional assessment plan as "an unnecessary, 
intrusive, bureaucratic activity." 

Resources. Second to faculty resistance as the barrier to implementation of outcomes 
assessment plans was resources. Over 26 institutions responded that resources, mainly financial, 
were an impediment to beginning the assessment process. A two-year public community college 
stated that lack of human and financial resources was critical. Another two-year college cited 
decreased state and local funding sources. One public comprehensive university stated 
succinctly that the major difficulty was cost. Another institution, a public, comprehensive 
university from New York was more pessimistic, "mostly funding, a problem that we will not 
overcome". And finally, a private liberal arts college related that the problem it had in this 
regard was lack of staffing, time, and money for assessment. 
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Overcoming the Barriers. A major public research university identified barriers to institution- 
wide outcomes assessment and spoke of how their institution is addressing those barriers. The 
difficulties identified by this institution are ones that many large complex universities might 
encounter. While the institution claims that it does not have an institution-wide plan, it indicates 
that it has outcomes assessment embedded in academic programs. One barrier has been the 
demand on leadership and faculty to concentrate on other institutional priorities and 
accountability mandates. The recision of state funding has had an impact on budget priorities. 

In addition, the decentralized structures of the institution make an institution-wide plan more 
complicated. 

The university is overcoming these difficulties. For example, the state is releasing funds for the 
assessment of learning outcomes, has appointed an associate provost for planning to develop and 
implement a university-wide assessment plan, and has established an Assessment Advisory 
Committee to oversee university- wide assessment activities. 

Many institutions shared the ways in which they were attempting to overcome the barriers 
previously-discussed. A private liberal arts college in New York established faculty committees 
and participates in Pew Roundtable discussions as a way to engage faculty. A private two-year 
college in Pennsylvania sends a faculty representative to a conference of the American 
Association for Higher Education to gather relevant materials for distribution to all faculty. 

This representative is a senior professor who also chairs the internal faculty outcomes assessment 
committee. A public two-year college in Pennsylvania indicates that it has asked faculty to 
develop syllabi that include outcomes, and this institution now provides in-service workshops on 
writing outcomes statements. 

A private comprehensive university reports that originally its efforts in outcomes assessment 
were decentralized, but they are now being directed by a single individual with widespread 
support from other members of the university community. 

A public two-year institution with dwindling funding pointed out that it has increased its efforts 
to obtain external funding. Likewise, another two-year public college strengthened its Office of 
Planning and Institutional Research to enhance outcomes assessment activities. A private 
comprehensive university in Pennsylvania is redesigning administrative and academic structures 
to accommodate institutional research, data collection, and the implementation of an outcomes 
assessment plan, and it has revised its institutional priorities. At a private liberal arts college, the 
director of academic assessment has formed an assessment leadership team which includes a 
liaison to academic deans. A private comprehensive university is considering establishing a 
system of incentives for those who engage in outcomes assessment, and a public two-year 
college in New Jersey intends to provide faculty release time, hire consultants, and seek grant 
funding as a means of gamering resources for outcomes assessment. 




ERIC 



1995 Outcomes Assessment Survey 



17 



Institutions with a Plan 

Faculty. Institutions that have outcomes assessment programs in place report that issues relating 
to faculty involvement are the primary barrier to sustaining assessment activities, although they 
reported this barrier less frequently than institutions beginning the outcomes assessment process. 

For a public two-year college, the issue was: "Faculty anxiety about being evaluated. We have 
tried to move ownership of the process and data to faculty, and we also routinely reassure them 
that the purpose of outcomes assessment is program evaluation, not faculty evaluation." 

A public comprehensive university in New York stated: “There is faculty resistance 
(accountability and trust issues). We have developed a process that faculty/departments all are 
required to conduct assessment, follow the college assessment guideline, show evidence that 
program improvement/changes are considered based on the assessment data.” 

At a public liberal arts college in New Jersey, it was reported that faculty, in particular, have 
resisted quantification and applying across-the-board instruments to all students. “We have 
encouraged qualitative research and we have given programs great scope in tailoring instruments 
to meet their own needs in order to facilitate the assessment process.” 

Resources and Time. Not surprisingly, many institutions also identified financial and human 
resources, as well as time, as barriers to overcome, even though their assessment plans have been 
implemented. 

A private, doctorate-granting institution in New Jersey stated the following: "Time and 
resources — not enough of either. Overcome somewhat in past few years by designation of an 
academic administration to oversee it and beefed up Institutional Research staff. The institution 
does not favor a highly bureaucratized, mechanistic assessment program. We regard our 
students' successful outcomes in the real world as more salient than portfolios. 

A private liberal arts college in Pennsylvania cited time constraints on a very busy faculty as a 
barrier, along with a lack of awareness of assessment and its value, and the need for individual 
faculty to develop effective self assessment tools. Their efforts however, will result in a course- 
integrated model for the use of portfolios in the humanities, history, and social sciences in fall of 
1995. 

A public two-year college in New York saw as the main barrier, "[T]ime commitment. The 
process is a complex, time consuming one which extends over a period of years. It is difficult to 
maintain focus and ongoing motivation. Also, certain degree programs are more readily assessed 
and measured in terms of student outcomes than others. While progress has been realized in 
rewriting all program objectives and course objectives, measurement/appraisal instruments need 
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further attention. 

Overcoming the Barriers. Many institutions stated ways in which they have attempted to 
overcome the difficulties and barriers to the successful implementation of assessment programs. 

To overcome the barrier of human resources, a private specialized college in Pennsylvania stated: 
"Finding the needed human resources has been very difficult, because faculty and staff are 
limited and outcomes assessment can be quite labor-intensive. The primary strategy has been to 
ease into the process, adding one piece at a time and allowing sufficient time to adjust to each 
new addition. 

A public comprehensive university in Maryland cited the lack of an assessment infrastructure, 
pointing out that the provost established a university Assessment Office, staffed with an assistant 
vice president; a University Assessment Council, which established assessment coordinators at 
the department level; and university-wide committees on assessment in general education and the 
majors, of which administrators are ex-officio members. The institution has worked to change 
the perception that the focus on assessment was solely for external reporting. The approved 
philosophy is that the purpose of the assessment program is to improve teaching and learning. 
Therefore, all policy statements and 21 "University Guiding Principles for Assessment" are 
based on this philosophy. To boost faculty morale, the University Assessment Council, in 1995, 
established Excellence in Assessment Awards. 

Another public comprehensive university, this one in New Jersey, is beginning to overcome 
faculty resistance through assessment principles and practices that emphasize faculty initiative 
and ownership of the process and through more visible and tangible administrative support. 
According to a private specialized institution in Pennsylvania, faculty resistance has been 
overcome by making outcomes assessment part of program review, by incorporating program 
review into the university's governance structure, and by offering educational programs such as 
workshops. 

Some wondered how outcomes assessment may be implemented at a multi-campus institution. 

A public two-year college in New York noted that it is a three-campus system, with each campus 
independently accredited and distinctively different in its age, stage of development, and service 
area characteristics. Such an institution represents both logistical and philosophical problems, 
which have been managed by having a single, college-wide assessment team, coordinated by a 
central academic vice president who develops and oversees the program. 
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A Role for CHE 

Institutions without a plan were asked: How might CHE best assist your institution in the 
development of an institution-wide outcomes assessment plan? Institutions with a plan were 
asked: What assistance might CHE/MSA provide in support of your institutional assessment 
efforts? The open-ended responses offered suggestions for Commission materials that offered 
examples of assessment plans and their application to specific types of institutions, Commission- 
sponsored workshops, and the publication of standards, policies and/or guidelines. 

Institutions without a Plan 

At least 47 institutions without a plan (3 1 percent of respondents) requested that the Commission 
provide examples of effective outcomes assessment plans. These institutions want to review 
plans from other institutions as they develop their own plans. 

The next most frequent request was for the Commission to sponsor workshops, conferences, or 
seminars on a routine basis. 

Several institutions identified the need for the Commission to be clearer in its expectations of 
institutions regarding outcomes assessment. For example, a two-year public institution in 
Maryland called for specific requirements for (1) institution-wide outcomes, (2) program and 
course level outcomes to gauge effectiveness, (3) specific guidelines for learning outcomes, and 
(4) a policy statement or other public position on standardized testing. 

In another instance, a public comprehensive university in New Jersey was more emphatic: 
"Require it.. .Force the issue, and we will have to respond!” Yet another view came from a 
private liberal arts college in New York: "If you want it, mandate it in clear and straightforward 
terms so that institutions can move forward, rather than wasting time trying to understand what 
needs to be done." 

Other suggestions ranged from an institution in Puerto Rico requesting documents be printed in 
Spanish, to other institutions asking the Commission to provide handbooks or guides and to 
identify consultants who can assist the institutions. 

Institutions with a Plan 

It appears that even institutions in the midst of outcomes assessment want better clarification of 
the Commission’s expectations for outcomes assessment. In addition, they want the Commission 
staff to share information about exemplary assessment programs and to sponsor workshops, 
conferences, and seminars on outcomes assessment. 
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Some institutions with a plan do not feel that the Commission is as clear as it should or could be 
in its expectations for outcomes assessment. This was the view of a private, proprietary 
institution in Pennsylvania that asked that CHE "clearly communicate your expectations 
regarding assessment standards." A public two-year college in New York stated that the 
Commission needs to be more insistent on comprehensive outcomes assessment as a primary 
criterion for Middle States re-accreditation. A major appeal by the Commission to academic 
leadership would provide more visible support — philosophically, structurally, and 
financially — to assessment and might enhance its status as an integral component of planning 
and policymaking. 

Another way the Commission might be helpful, according to a public comprehensive institution 
in New York, would be to define the Commission's concept of outcomes assessment — in terms 
of goals, methodology, and focus — and make assessment a formal requirement of accreditation. 

A private doctorate-granting university in Pennsylvania made the plea for the Commission to 
promote an understanding throughout the higher education community that a complex, 
comprehensive university needs to have a variety of assessment modalities, rather than a 
uniform, unilateral approach to program assessment. Programs for full and part-time students, 
for undergraduate, graduate, and post-graduate students, and with credit and non-credit offerings 
require different approaches to assessment within the same institution. 

Institutions without a plan, as did those just beginning the assessment process, also felt that the 
Commission could be helpful by providing conferences, workshops, and seminars on outcomes 
assessment, and by publishing handbooks, guides, and databases that provide current information 
on outcomes assessment, including methods and instruments that might be used. There were 
many calls for examples, examples, examples! This perspective was captured in the suggestion 
from a public, comprehensive university in Pennsylvania: The Commission can provide support 
in a variety of ways, including: (1) emphasizing the importance of outcomes assessment in 
Middle States reviews and using data collected to serve our clients better; (2) providing 
Commission-sponsored workshops to promote outcomes assessment and to showcase good 
programs within the region; (3) providing limited consultation on outcomes assessment to 
universities, or at least helping them to identify consultants who can help; and (4) recognizing 
exemplary programs within the region. 
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Conclusion 

The findings from the 1995 survey on outcomes assessment in the Middle States region provided 
several new insights and validated several previously held views that were based on an intuitive 
understanding of outcomes assessment as it is practiced in the region. These findings are based 
on both the quantitative data and responses to open-ended questions. 

A Quantitative Snapshot 

The survey revealed that over one-half of the institutions in the region have no institution-wide 
plan for outcomes assessment, and over one-half of those that do have a plan are public 
institutions. Over one-third of the institutions have had an assessment plan in operation for three 
or more years. 

Most of the institutions that have a plan utilize some of the more direct measures of institutional 
effectiveness, but few emphasize direct measures of teaching and learning. Data are most 
routinely gathered on enrollment, retention, graduation, and attrition rates, but few concentrate 
on using the results of discipline content exams, basic skills, higher order skills, or student 
portfolios in an outcomes assessment program. Furthermore, while most institutions utilize 
nationally-normed tests, there is a significant interest in including locally-developed instruments 
in assessment programs. 

The majority of institutions without a plan have completed no more than two or three of the nine 
steps considered prerequisite to developing a plan. The steps that have been completed generally 
involve some of the more mechanical tasks, such as developing inventories of instruments in use, 
but institutions generally have not tackled the more thorny issues of evaluating the effectiveness 
of those instruments or deciding which outcomes still need to be measured, beyond those that are 
already being measured. 

Only a handful of institutions with a plan indicated that their assessments have led them to 
modify their goals and objectives comprehensively, although most have done so moderately. 

A large majority of institutions noted that they communicated assessment results to the campus 
by formal report or reports, while some did so only informally or selectively or in response to 
individual requests. 

Faculty tend to be fully involved in developing assessment plans at most institutions, while 
academic administrators tend to be fully involved in the continuous administration of those 
plans. Faculty committees, rather than individual faculty, are deeply involved in analyzing 
assessment results, along with academic administrators. However, the participation of faculty 
and faculty committees declines significantly at the largest institutions, where the responsibility 
shifts to academic administrators and institutional researchers. 
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Perspectives on Implementation 

Open-ended responses to questions on barriers to implementation and how they can be overcome 
indicated that the chief barriers to implementing new assessment activities or sustaining existing 
intitatives are faculty and resources. In addition, the Commission does have a role in promoting 
assessment within the region. 

Faculty indifference, if not resistance, appears across institutional types, whether public or 
private, from research universities to liberal arts and community colleges. Faculty are viewed as 
generally suspicious of this activity, and it is a challenge for administrators and others to engage 
faculty successfully in the assessment process. 

The next most often cited barrier to outcomes assessment is resources, both human and 
financial, with financial mentioned more frequently. Many institutions, especially in some 
states, are grappling with dwindling financial resources, and they maintain that these cutbacks 
are an impediment to outcomes assessment activities. Others cite limited staff and other 
personnel constraints as barriers. Nevertheless, most institutions have attempted to overcome the 
issue of resources and have begun to establish or are contining outcomes assessment within the 
limits of their resources. 

Institutions indicated that they would like the Commission to assist by providing examples of 
effective assessment programs. While the Commission expects to satisfy the requests for 
workshops or seminars in the immediate future, a critical mass of information about programs 
that would be suitable as models is not readily available, and Commission staff will need to 
identify such examples. Institutions also should understand that the design of an outcomes 
assessment plan is unique to each institution, and a plan from another institution may not be 
appropriately utilized on another campus with significant adaptation. Each institution must 
conceptualize its own outcomes assessment program, based on its mission, goals, objectives, and 
resources. Staff also may be able to provide examples of institutions that have established cost- 
effective programs with limited resources. 

Some institutions suggested that the Commission should clarify its expectations for outcomes 
assessment, and the Commission is developing both a policy statement and guidelines on the 
subject. Some respondents even requested the Commission to be more prescriptive in its 
expectations, although this suggestion cannot be accommodated within the letter or spirit of 
Characteristics of Excellence (1994), which is the Commission’s primary statement of standards. 

This survey indicates that outcomes assessment is of great importance to institutions in the 
Middle States region, they currently are in various stages of developing and implementing 
outcomes assessment plans, many institutions are working to overcome barriers to 
implementation, and all institutions need assistance from the Commission in this endeavor. 
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Recommendations 

Based on the findings from the 1995 Outcomes Assessment Survey, Commission staff makes the 
following recommendations: 

1. The revised Framework for Outcomes Assessment should be advertised widely to 

institutions in the region, emphasizing its guidelines for approaching conceptual 
issues in the development of outcomes assessment plans. 

2. The Commission should sponsor as many seminars as possible in the coming years to 

assist institutions with specific strategies for (a) completing the nine preliminary 
steps for developing a plan; (b) collegially developing a plain on campus; (c) the 
continuous administration of assessment plans; and (d) post-assessment strategies, 
involving incorporating assessment findings in long-range institututional planning 
and the self-study process as well as communicating those findings to all of the 
constituencies at various types of campuses. 

3. Commission staff should provide Chairs and members of evaluation teams attending 

Commission-sponsored workshops with an update on the state of assessment in 
the region, how they should place the institution they are visiting in a framework 
of readiness to develop a plan or of effectiveness in administering an existing 
plan, and how best to evaluate what an institution realistically might be expected 
to accomplish in the five years between an evaluation visit and the following 
periodic review report. 

4. The report on the 1995 Outcomes Assessment Survey should be published and 

disseminated to the membership and to selected constituents and interested 
organizations. 
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